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HANDHELD PORTABLE DIGITAL 
GEOGRAPHIC DATA MANAGER 

CROSS-REFERENCE TO A RELATED 
APPLICATION 

This application claims the benefit of the filing date of 
U.S. Provisional Application Ser. No. 60/025,528 filed on 
Sep. 6, 1996. 

BACKGROUND OF THE INVENTION 

It is often necessary for insurance adjusters, 
geophysicists, construction workers, real estate developers, 
and others working in the field to gather and assimilate 
geographic, topographic, visual (i.e., photographic),and 
other information about a site. These sites may be in 
locations which are remote and/or have no readily available 
access to computers or even power sources. Currently, there 
is no handheld portable device for gathering and processing 
such information. 

Printed maps are available for viewing almost all features 
that have geographical locations. A limitation of printed 
maps is that a user must manually sort through the entire 
map in order to find the relevant geographical features. 
Electronic maps have recently become available to replace 
paper maps for some applications. A map database is stored 
in a memory storage device as a bit map or as vectors that 
point to a map character. 

Many electronic location determination systems are avail- 
able or have been proposed to provide electronic location 
information to a user equipped with a location determination 
receiver. Ground-based location determination systems are 
well known and include systems that were developed pri- 
marily for communications, such as cellular telephone, FM 
broadcast, and AM broadcast. 

The Global Positioning System (GPS) is a global navi- 
gation system that enables the user to utilize signals broad- 
cast from satellites in order to identify positions. GPS 
systems are well known and widely used to accurately locate 
specific positions. GPS systems, which utilize a network of 
satellites to identify locations on the ground, are widely use d 
in automotive and nautical naviga tion, construction, mining, 
and farming, as well as in a~varietv of other applications. 
"See, forexample, U.S. Pat. No. 5,528,248 and PCT Appli- 
cation No. WO 95/05686. Other satellite positioning 
systems, such as the Global Orbiting Navigational System 
(GLONASS) are also known. Cartographers, utilities 
operators, wildlife managers, and others have used GPS 
technology in conjunction with Geographical Information 
Systems (GIS) to augment map making procedures and 
other field work, particularly in remote locations. In such 
applications, there has been an increasing use of computer s 
4p facilitate data storage and retrieval. 

Personal Digital Assistant (PDA) is a generic name for a 
hand held pe rsona l computing device having a volume in the 
Tange of about 2 (JO to 1200 cubic centimeters. ruAs can 
have as much computing power as some desk top -person al 
computers and have been used in a wide variety of 
"applications, inclu ding mapping . 

Unfortunately, hardware and software hurdles have lim- 
ited the ability to utilize multicomponent data gathering 
systems in the field. Connecting equipment such as a GPS, 
a two-way radio, and a handheld pen computer for use in the 
field poses a number of obstacles. Many wires and cables 
work fine on a desktop computer but not on a handheld 
device. The essence of the subject invention is the devel- 
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opment of a rugged, fully-integrated, easy to use handheld 
m ulticomponent field dat a gathering system for gathering. ' 
positional, image, and other data, and simultaneously pro- 
cessing this information. 

5 BRIEF SUMMARY OF THE INVENTION 

The subject invention concerns methods and devices 
which aid in collecting and verifying planning and engi- 
neering field data. The devices of the subject invention, 

10 which can be handheld, portable, and digital, can be referred 
to as Geographic Data Managers (GDMs). The GDMs of the 
subject invention are designed to meet the needs of, for 
example, the construction and utility industries. The meth- 
ods and devices of the subject invention are also applicable 

15 to other tasks such as environmental sampling, surveying , 
agricultural field data collection, property appraisal, law 
enforcement, and construction inspe ction. Specific app lied 
uons include tne gas, water, sewer, telephone, and cable TV 
utilities; local, state, and federal government inspection 

2 q agencies; and the insurance and transportation industries. 
The GDMs of the subject invention comprise multiple 
components interfaced with, for example, acentral process- 
ingunit (CPU). Preferably, these multiple components 
include ar leasi one digital camera, at least one GPS receiv - 

25 ing anten na, a (jFS processing component, wireless radio 
c communication capability, a digital compass with 
inclinometer, and user interaction capability. In a preferred 
embodiment of the subject invention, the GDM comprises at 
jeast two digital came r asjwhich facilitate the determi nation 

30 _ot the distance fromthe GDM to an objec t. In a specinc 
embodiment, a digital compass and an inclinometer of the 
GDM can enable the determination of the orientation of the 
camera so that the exact location of an object which appears 
in the images from the digital cameras can be readily 

35 determined. 

The subject GDM can be utilized in applications where 
the determination of an object's location in two-dimensions 
is required, for example when the height of the object is not 
needed. In a specific embodiment, the subject GDM can be 

40 equipped with two GPS antennas and two cameras which 
can allow the determination of the object's location in two 
dimensions. In this embodiment, the two GPS antennas can 
be offset such that a heading and position of the GDM can 
be determined and the two digital cameras can provide the 

45 distance to the object, thus providing data needed to deter- 
mine the object's location in two dimensions. Alternatively, 
a compass can provide the heading and two cameras can be 
used to provide the distance to the object from the location 
of the GDM. In another alternative embodiment, for 

50 example, if the direction of an object is known, an incli- 
nometer and the two digital cameras can provide pitch and 
distance data which can be used to determine the height and 
distance to the object from the location of the GDM. 
In a preferred embodiment, the subject GDM can com- 

55 prise at least three GPS antennas and corresponding 
receivers, wherein the GPS antennas are in a plane. The 
receipt of the signal from one GPS satellite by all three of the 
GPS antennas can provide information with respect to 
heading, pitch, and roll of the GDM, as well as location. 

60 Accordingly, when this information is combine d with the 
d istan ce information dete rmined from t fc rcn mages from the a t 
least two camera's", the UUM can provide information as to 
" the uxacHocatio n, in three dimensions, of the imaged object. 
Advantageously, the combination of the three GPS antennas 

65 can provide a more accurate determination of the GDM 
location due to receipt of the GPS signals by all three GPS 
antennas in a plane. 
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In another specific embodiment, the subject GDM can (20) ethernet; 
comprise at least one gyroscope, for example a laser (21) PC card; and 
gyroscope, which can provide data related to the GDM's , 2 2) touch/pen input, 
heading, pitch and/or roll. Advantageously, a laser gyro- 
scope can be less susceptible to confusion by metals than a 5 BRIEF DESCRIPTION OF THE DRAWINGS 
compass, In addition, a gyroscope can be calibrated from 

data received from the GPS when the GDM's GPS antenna FIG. 1 is a block diagram showing the components of one 

is moving. embodiment of the device of the subject invention. 

In a further embodiment, the subject GDM can be incor- FIG. 2 is a schematic showing the determination of the 

J J/ porated with another structure, for_example a vehicle. This ™ distance to an object from the device of the subject invention 

other structure can incorporate at least one GPS antenna and using the images produced by two digital cameras, 

a means for providing data to the GDM with respect to the FIG. 3 is a schematic of the embedded controller proces- 

GPS antenna's position relative to the GDM's camera(s). sor system. 

This embodiment can be useful when the GDM is used, for F IG. 4 is a schematic of the digital compass and incli- 

example near tall buildings, where the GPS satellite signals 15 nome t e r 

can bounce off and/or be blocked. Tto GDM can also have FIG. 5 is a schematic of the flash memory subsystem. 

a means for mounting to the structure, for example a vehicle, . J 

such that its location relative to the structure is known. In a FIG * 6 15 a schematic of the GPS receiver subsystem, 

specific embodiment, the GPS antennas can be mounted on FIG. 7 is a schematic of the LCD display subsystem, 

the structure in order to allow a greater distance between the 20 FIG. 8 is a schematic of the touch screen interface and 

antennas and, therefore more accurate distance determina- module. 

tl0ns * FIG. 9 is a schematic of the stereo digital camera sub- 
In a preferred embodiment of the subject invention, the system. 
f L GDM co mprises capability for wireless transmission 26 o f piG. 10 is a schematic of the digital stereo range finder 
data to a host computer at a remote location , 'rhe <JL)R~of subsystem. 

T^^^^aanb^^^^A^ pig. 11 is a schematic of the voice command subsystem. 

controller 28 and/or a voice controller 30. Preferably, the . J 

G DM 6t the subject invention farther com prises a power 12 » a schematic of the wireless network interface, 

source. This power source may be, for example, a battery FIG. 13 is a block diagram of an embodiment of a GDM 

pack and any necessary electrical components to modulate 30 in accordance with the subject invention, 

the current so that it is compatible with the requirements of FIG. 14 is a diagram of object oriented control of a camera 

the various data gathering and processing components. subsystem as a JAVA native method, in accordance with the 

In a preferred embodiment, the integrated handheld field subject invention, 

digital data mapping device of the subject invention is 35 FIG. 15 is an illustration of a handheld GDM, in accor- 

capable of: dance with the subject invention. 

(1) collecting and mapping global position system (GPS) FIG. 16 is an illustration of a GDM which can be 
data; - - incorporated with a vehicle, in accordance with the subject 

(2) reporting GPS location, speed, and direction in a invention, 
seamless fashion; 40 FIG. 17 shows the basic functional modules of a GDM, in 

(3) reporting orientation including magnetic direction, accordance with the subject invention, 
pitch, and roll; FIG. 18 illustrates a camera module of a GDM, in 

(4) collecting multiple pairs of color, or grey level, digital accordance with the subject invention, which utilize a 
images (photographs) used to measure distance to an Dycam digital camera with two CCD sensors, 
object; 45 FIG. 19 illustrates a camera module of a GDM, in 

(5) voice command and voice query; accordance with the subject invention, which utilizes mir- 

(6) touch sensitive icon control; rors t0 relav 40 ^ hom me left °P tics and an ima S e from 
... j . , i • ,eu i_ me right optics to a high resolution CCD detector. 

(7) pr pvidinp icon-d nven n otes relating to field oh serva- & , & . 
t * ons T 50 FIG. 20 illustrates an embodiment of a GDM, m accor- 

/ott ~- — ^ . . j f ....... dance with the subject invention, which utilizes two GPS 

^Son Way * antennas and ^ image gatherin8 optics - 

/fVk ! 4 . • • „ i • j FIG. 21 illustrates an embodiment of a GDM, in accor- 

(9) mapping the present environment in points, lines, and , . A . . . • . 
V . dance with the subject invention, which utilizes a gyroscope 

area ea ures, ^ to p^^^ £ j ata re i a ted to the GDMs heading, pitch, and/or 

(10) providing navigation assistance to a desired location; T0 \\ 

(11) power management to allow a battery pack to provide 
power to various components; DETAILED DISCLOSURE OF THE INVENTION 

(12) collecting infrared images; The subject invention provides a system design, materials 

(13) bar code reading; 60 and methods which are highly advantageous in collecting 

(14) voice/audio annotating; and utilizing field data for use in a wide variety of applica- 

(15) video annotating; tions * ^ Geographic Data Manager (GDM) of the subject 
Ti/T-rt/r.u * I a *■ invention is a multi-component system for gathering 

(16) WEB/Ethernet phone function; . . j *u j . j • u * i 
) ' . . positional, image, and other data, and simultaneously pro- 

(17) fingerprint recognition; 65 cessing ^ ^formation. sub j ect invention is also appli- 

(18) digital inking; cable to other tasks such as construction and utility 

(1 9) native methods/Applets as internet compliant design; monitoring, environmental sampling, surveying, agricultural 
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field data collection, law enforcement, public safety, and 
property appraisal Specific applications include the gas, 
water, sewer, telephone, and cable TV utilities; local, state, 
and federal government inspection agencies; and the insur- 
ance and transportation industries. 5 

Referring to FIG. 17, a block diagram is shown illustrat- 
ing the main functional blocks of the subject invention. In a 
preferred embodiment, the GDM of the subject invention 
comprises a central processor unit (CPU) 1, preferably a low 
power microprocessor, which coordinates the transmission 10 
and receipt of data and commands from the other system 
components and processes the data gathered, stored, and/or 
processed, in order to provide data needed during the 
operation of the subject GDM. High speed data links 2 can 
link the central processor unit 1 to the camera module 3, a 15 
user interface module 4, a directional and orientation mod- 
ule 5, a location module 6, a power module 7, and an 
optional network communication module 8. 

The camera module 3 can ga ther data representing digital _ 
images of o bjects under study, ine camera module 3 can ^° 
receive control signals l iom the CI*U 1 and send d ata 
re presenting digital imagesto the CPU 1 lor storage anaVor 
processing. The user interface module 4, connected to the 
CPU 1 by a high speed data link 2, allows a user to interface 
with the GDM. The user interface 4 can allow a user to view 25 
images stored in the GDM, an object currently in the 
camera(s) view, and/or icons which can allow the user to 
control the operation of the GDM. The user interface 4 can 
utilize voice recognition and commands, a heads up display, 
V. computer display flat panel with pen or touch input, and/or 
various other user interfaces. 

The directional and orientation module 5, connected to 
the CElL Lby a high sp e ed data link 2, supplies data to th e 
CPU 1 wfrjch is usedlo determine the heading, pitch, and/o r 
rol l of the GDM and can also receive data from the CPU 1. 
FojLexample, the CPU 1 



^ can sen d data which the CPy 1 

receives trom tne location module 6, which can be useful i n 
calibration of the equip ment ot tne directional and orienta- 
tionmodu le 5. Compasses, inclinometers, gyroscopes, and/ 
or combinations of GPS receivers can be utilized for col- 
lection data which is relevant to the direction and orientation 
of the GDM. 

The location module 6, connected to the CPU 1 by a high 
speed data link 2, gathers and sends data to the CPU 1 which 
can be utilized to determine the location of the GDM. The 
location module 6 can utilize a variety of systems to gather 
this data, including, for example, satellite and/or ground 
based positioning systems. In a preferred embodiment of the 
subject invention, the location module utilizes GPS antennas 
and receivers 32 to receive satellite signals which allow the 
CPU 1 to determine the position of the GDM. In specific 
embodiments, the directional and orientation module 5 and 
the location module 6 can be combined, for example when 
three GPS antennas and receivers 32 are utilized to provide 
data relevant to position, pitch, and/or roll of the GDM. 

The optional network communication module 8, con- 
nected to the CPU 1 by a high speed data link 2, can be 
utilized to enable the GDM to communicate with remote 
facjlitjg^Joi ^example a home office computer which can 
receive and send oa t Ttrom and to the CjDM The network 
communication module 8 can utilize, lor example, wireless 

wired networks . In a specific embodiment, at least two 
GDMs can communicate with each other via network com- 
munication modules, for example, to transfer data. 

The handheld data gathering device of the subject inven- 
tion can, in a preferred embodiment, comprise: 



30 



35 



40 



45 



50 



55 



60 



65 



(a) An imbedded computer capable of storing and pro- 
cessing image data add interlacing to multiple" data 
sensing modules. The user sees the graphical and alpha 
numeric re sults on a LCD_or appropriate display with 
p en and/orjtpuch screen input, eith er coloror black an d 
wEiteVThe user controls the system through touching or 
pointing to action icons or via voice commands. 
Alternatively, a mouse and keyboard may be used for 
more conventional user input and control of the system 

(b) A global position receiver (GPS) 32 capable of accu- 
rately capturing geographic location with post process- 
ing or real-time differential correction capability. Better 
than one meter accuracy can be achieved with such a 
system. An interface from the GPS to the centra l 

A i computer to process the location data and th e necessary 
fAAsaFtware to display and record the results . TJiffSfTntial 
correction of GPS data is used to improve the accuracy 
either real-time or post processing as available. 

(c) A digital compass 34 capable of providing accurate 
heading. Other devices capable of providing heading, 
for example, a gyroscope 38 or a pair of GPS receivers 
32, can be used to calculate heading. A gyroscope 38 
can be calibrated from heading obtained from the GPS 
receiver when the GDM is moving. In a specific 
embodiment, a digital compass can be utilized with an 
error of less than about one degree. An interface from 
the digital heading device to the central processor and 
software capable of processing and ^dispjaying and 
storing the direction or heading informatioiL ~ 

(d) Means capable of providing accurate pitch and roll 
data, for example digital inclinometers 36. Accurate 
pitch and roll data, for example less than one degree of 
error, for a plane parallel to the center line of the 
camera's field of view can aid in resolving the exact 
location of the object under observation. For example, 
available digital inclinometers can be utilized with an 
error of less than about one degree. A gyroscope type 
device and/or a set of GPS receivers can also be used 
to collect real-time orientation data. The interface from, 
for example, an inclinometer, to the imbedded com- 
puter transfers the pitch and roll data. The system 
software is responsible for processing, displaying and 
storing the pitch and roll data. It also makes the data 
available for use in calculating new coordinates. 

(e) A pair of digital cameras 40 with matched optics 
mounted with the lenses in the same plane and with the 
line through the center of view of each lens parallel to 
the line through the other. Anj nterface from the cam - 
as to the imbedded computer processor and s oftwa re 

capable of displaying, processing, and storing the 
^Jges. 

(f) Stereo image processing software or hardware algo- 
rithms capable of locating a selected object in one 
image in the stereo image pair. Once the object is 
located in the stereo pair image the displacement 
between the two objects is used in conjunction with the 
optical characteristics of the lenses and the spacing of 
the lenses to calculate distance to the object of interest. 
Accuracy of plus or minus one inch can be achieved at 
fifteen feet, and about plus or minus a foot at thirty feet 
can be readily achieved using lenses which are only 
about nine inches apart. The distance accuracy drops 
off as the distance increases; however, longer range 
accuracy is achieved by sliding the cameras out to a 
wider spacing and/or attaching telephoto lenses to the 
cameras. For example, a 2x telephoto lens makes the 
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accuracy plus or minus one inch at 30 feet. The GDM cameras. The distance determination can be carried out by 

can be made with lenses spaced at nine inches to keep the GDM utilizing a software program which compares the 

the handheld unit small; however, when extended to 18 images produced by the two cameras in order to determine 

inches the accuracy is increased. The GDM can have a the distance apart a single object is in the two images, 

horizontal resolution of 640 pixels. Image sensors with 5 Referring to FIG. 2, this distance apart in the images is 

a higher resolution also increase distance accuracy. For directly proportional to the distance that the object is from 

maximum distance accuracy a higher resolution sensor the GDM. The distance to an object can be determined from 

is used in conjunction with widely spaced lenses and the following relationship: 
telephoto lenses 

(g) In addition to simple distance measurement and range 10 fd c 
finding to locate an object of interest to the user the pair d °~ ~d[ 
of stereo images collected can be used 25 to measure 

objects of interest in the field or at a later time by , 

storing the optical characteristic of the lenses and the where d * 18 the distance t0 we ob J ect > 

lenses spacing with the stereo pair of images produced. 15 d c is the spacing between the cameras, 

(h) Data stored in the system in the PC Card or appro- d,- is the offset of the two images of the object from the 
priate computer storage medium can be sent to a base cameras, and 

unit via a wireless network which can be made up of f is the focal length of the cameras' lenses, 

two-way digital packet radios or analog or digital cell Through a series of straightforward computational steps, the 

phones. Alternatively, data can be exchanged via an 20 GDMof the subject invention analyzes the distance between 

infrared link, a serial cable, a modem on the phone the object's images and converts this to the distance from the 

system, a wired network connection, Ethernet, a PC GDM to the object. 

Card, a floppy disk, or other available channels. i n a preferred embodiment, the subject GDM can have 

(i) Voice communications between units can be provided stereo camera lenses which are precisely rotated on an axis 
over the digital data link to aid in coordination activity 25 through a tine through the center of the optics of each lense. 
of the users in the field. Rotation of the lenses such that each points toward the object 

(j) The software of the prototype system is designed to under study can allow a more precise determination of the 

enhance the use of the Internet base world wide web. A distance to an object. In this embodiment, the first image set 

real time operating system can be used to manage each can be taken with the lenses parallel The distance to the 

of the devices such as the cameras 40; GPS or multiple 30 object can be calculated and this distance used to compute 

GPS receivers 32; the direction and orientation devices; the angle of rotation for each lense. The two images of the 

the wireless or wired network interface, the battery and same object are then matched by small changes in the angle 

power management subsystem; the graphics display; of rotation. The final range can be calculated utilizing the 

and the audio input and output devices. In the prototype angle of rotation of the lenses. 

system Java native methods are provided for each 35 In a preferred embodiment, the GDM simultaneously 

device. For example there is a Camera object which records the pitch and roll of the GDM as determined by, for 

provides the Web based Java program with stereo example, an inclinometer, and it records the magnetic direc- 

digital image pairs and range finding calculations. tion of the GDM as determined, for example, by a digital 

(k) A power management system designed to maximize compass. This information, together with the GPS 

the usable battery life of the GDM by regulating the use 40 information, are all processed essentially simultaneously by 

of power by the main computer system as well as each the CPU and can be, for example, transmitted to a remote 

of the data collection subsystem devices. host computer by wireless communication components. 

In a preferred embodiment, the GD M of the subjec t The following two tables illustrate the accuracy of dis- 

in vention utilizes a pair of digital cameras 40 which are n ot tance determination utilizing two specific embodiments of 

'only able to prov ide photographic imag es of sites of interest the subject invention. In these tables, one pixel depth gives 

but also to iacmiaie the dmerrn^naiinn or the distance tro m the range expected error for one pixel alignment accuracy of 

the GDM t o any object in the images produced by the digita l the image. 

TABLE 1 



Range To Target 
In Meters 


Image target 
offset in mm 


Image target 
offset in pixels 


One pixel View frame in 
depth Meters Meters 


3 


2 


363 


0.008 


0.88 by 0.60 


5 


1.2 


218 


0.021 


1.45 by 1.1 


10 


0.6 


110 


0.090 


2.9 by 2.2 


25 


0.24 


43 


0.51 


7.5 by 5.5 


50 


0.12 


22 


2.0 


14.5 by 11 



TABLE 2 



Range To Target Image Target Image Target One Pixel View Frame 

In Meters Offset in mm Offset In Pixels Depth Meters In Meters 

10 2.4 mm 436 0.0225 2.96 by 2.2 

25 0.96 mm 175 0.1273 7.5 by 5.5 
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TABLE 2-continued 



Range To Target 


Image Target 


Image Target 


One Pixel 


View Frame 


In Meters 


Offset in mm 


Offset In Pixels 


Depth Meters 


In Meters 


SO 


0.48 


86 


0.50 


14.5 by 11 


100 


0.24 


22 


2.00 


29 by 22 



Table 1 is for 12 mm lenses set at 500 mm lenses spacing 10 
and a 0.0055 mm per pixel CCD sensor, which can be 
utilized in a handheld GDM in accordance with the subject 
invention. Table 2 is for 12 mm lenses set at 2 meter lense 
spacing and a 0.0055 mm per pixel CCD sensor, which can 
be utilized in a vehicle mounted GDM in accordance with 15 
the subject invention. 

A variety of options relating to imaging optics can be 
utilized in accordance with the subject invention. Referring 
to FIG. 20, a left camera sensor and 10 optics and a right 
camera sensor and optics 12 can feed two digital cameras, 2Q 
each having a sensor, or one digital camera having two 
sensors. Referring to FIG. 19, mirrors 14 can be used to 
guide the right 16 and left 18 images to a single high 
resolution CCD detector 20. Other variations can be readily 
utilized by one skilled in the art having the benefit of the 
instant disclosure. 25 

The GDM of the subject inv ention can further compo se a 
display screen and Ymearis' lorlhe operator to interact with 
the GDM . This means of interaction may be, for example, by 
a touch screen, by voice recognition, or by a traditional 
keyboard. 30 

Aperson skilled in this art having the benefit of the instant 
disclosure would readily appreciate that a variety of modi- 
fications could be made to the specific embodiments exem- 
plified herein without departing from the spirit of the instant 
invention. For example, the GPS function of the GDM is for 35 
the purpose of providing a location for the GDM. This 
function could be performed by other locating means 
including, but not limited to, other satellite positioning 
systems and ground-based systems. Certainly, when the 
device of the subject invention is to be used indoors or in the 40 
vicinity of tall buildings, it may be advantageous to use a 
ground-based location system rather than a satellite system. 
"-Also, the device could display a map which the user could 
touch or in some other way designate the location of the 
GDM. In yet another embodiment, the GDM may be 45 
stationary, in which case its position will be known. 

The subject GDM can be utilized in applications where 
the determination of an object's location in two-dimensions 
is required, for example when the height of the object is not 
needed. In a specific embodiment, referring to FIG. 20, the so 
subject GDM can be equipped with two GPS antennas 22 
and two cameras 10 and 12 which can allow the determi- 
nation of the object's location in two dimensions. In this 
embodiment, the two GPS antennas 22 can be ofiset such 
that a heading and position of the GDM can be determined 55 
and the two digital cameras 40 can provide the distance to 
the object, thus providing data needed to determine the Ui 
object's location in two dimensions. Alternatively, a com-**( 
pass can provide the heading and two cameras can be used 
to provide the distance to the object from the location of the 60 
GDM. In another alternative embodiment, for example, if 
the direction of an object is known, an inclinometer and the 
two digital cameras can provide pitch and distance data 
which can be used to determine the height and distance to 
the object from the location of the GDM. 65 

In a preferred embodiment, the subject GDM can com- 
prise at least three GPS antennas and corresponding 



receivers, wherein the GPS antennas are in a plane. The 
receipt of the signal from one GPS satellite by all three of the 
GPS antennas can provide information with respect to pitch 
and roll of the GDM, as well as location. Accordingly, when 
this information is combined with the distance information 
determined from the images from the at least two cameras, 
the GDM can provide information as to the exact location, 
in three dimensions, of the imaged object. Advantageously, 
the combination of the three GPS antennas can provide a 
more accurate determination of the GDM location due to 
receipt of the GPS signals by all three GPS antennas in a 
plane. 

In another specific embodiment, referring to FIG. 21, the 
subject GDM can comprise at least one gyroscope, for 
example a laser gyroscope 24, which can provide data 
related to the GDM's heading, pitch and/or roll. 
Advantageously, a laser gyroscope can be less susceptible to 
confusion by metals than a compass. In addition, a gyro- 
scope can be calibrated from data received from the GPS 
when the GDM's GPS is moving. 

In a further embodiment, the subject GDM can be incor- 
porated with another structure, for example a vehicle. This 
other structure can incorporate at least one GPS antenna and 
a means for providing data to the GDM with respect to the 
GPS antenna's position relative to the GDM's camera(s). 
This embodiment can be useful when the GDM is used, for 
example near tall buildings, where the GPS satellite signals 
can bounce off and/or be blocked. The GDM can also have 
a means for mounting to a structure, for example a vehicle, 
such that its location relative to the structure is known. In a 
specific embodiment, the GPS antennas can be mounted on 
a structure in order to allow a greater distance between the 
antennas and, therefore more accurate distance determina- 
tions. 

frnrifl * n F] n STlsrifl^ ^ r " h ^ H ' ryi ^" f of f h *> ci,h j ect 
invention is shown which can be used ig a vehicle sue 



uch a§, 

an automobile, boat, airplane, or helicopter. In this specific 
embodiment, two GPS antennas 22, two cameras 10 and 12, 
and a gyroscope 24 are incorporated into data bar 42 which 
can bejnounted 44 oa a vehicle by way, of a base mount. 
Advantageously, this can allow a large spacing between 
cameras and, therefore, very accurate distance determina- 
tions. The data bar 42 is connected to a user interface 46 by 
a high speed data link 48. This data link 48 can be wired or 
wireless. The central processomnit can be located within th e 
data bar with the user interface being a periphe ral 
a ttachme nt, ort he central processor unit can be in the u ser 
interfa^e ^unitTfie user interface unit can be mounted in th e 
v^Icie'tb^free a user's hands or can be held by a user dur ing 
use. Other combinations of components and mounting tech- 
mques can be readily designed and used by a person skilled 
in the art having the benefit of the instant disclosure. 

In an alternative embodiment, a handheld user interface 
unit can have a gyroscope and two cameras with a GPS 
antenna being attached to a vehicle. A user can position the 
vehicle such that a GPS signal is available to provide the 
location of the GDM. Once the GDM position is known and 
the gyroscope in the hand held unit is calibrated, the user can 
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approach objects in regions where a GPS signal may or may maps, and other data for verification and collection. See FIG. 

not be available, for example inside a building. The GDM 7. The second industry pack module is a storage module with 

can keep track of the movements of the GDM by data eight megabytes of flash memory. See FIG. 5. 

received from the gyroscope in order to assist in the deter- The user interacts via a DYNAPRO touch screen 

mination of the position of imaged objects. This can enable 5 (DYNAPRO SC3 Touch Screen Controller and Software 

the use of the subject GDM in environments where a useable Users Reference Version 1.1, 1995, DYNAPRO Thin Film 

GPS satellite signal is not necessarily available. In a specific Products) on the sixth serial port and/or via voice prompting 

embodiment, a set of accelerometers and a set of gyroscopes an d command provided by a OKI semiconductor voice 

can provide dead reckoning position calculation where a control module connected to the first serial port. See FIGS, 

reliable GPS signal is not necessarily available. 10 8 and 11. 

Another embodiment of the subject invention utilizes A stefeo k of GatQr Di ^, al (Qator 

more than two cameras in order to provide enhanced spatial Camera Users GufcJe an(J Reference) 1994> DYCAMj 

determination of an object. In another embodiment, the chatworth, Calif.) are attached to the second serial port via 

cameras of the subject invention can be moved to increase a hi h d synchronous co^i^ ^ ^ cameras 

the distance between the lenses so as to enhance the accu- 15 provi de stereo pairs of images for depth-of-fleld calculations 

racy of distance determinations, especially for objects an(J three-dimensional of objects of interest. See 

located at a substantial distance from the GDM. The move- p[Q$ 9 anc j 10 

able camera lenses can be retractable to their original ... . , ™, . . . . „™ 

position for convenience. The camera may utilize wide- , /^J?^ 5? ^ GPS receiver (Trrable GPS 

angle lenses for objects close to the GDM. 20 ^ ece ^ 1995 Tnmble Navigation, Sunnyvale, Calif.) ison 

Cameras and/or camera lenses which are based on differ- me third P 0 ^* 0 , 01 al correction receiver which 
ent parts of the spectrum, for example infrared, may also be « re f s GP JL data t0 1-2 jneter accuracy is connected 
utilized in accordance with the subject invention. In a directl y to the GPS receiver * See 6 
specific embodiment, the camera may include an infrared t A PRECISION NAVIGATION digital compass unit pro- 
sensor for heat detection. The GDM may be equipped with 25 vides direction, pitch, and roll, as well as temperature, on the 
additional chemical and environmental sensors to augment fourth P ort (PRECISION NAVIGATION TCM2 Electronic 
the type and amount of data which can be gathered. Compass, 1994, Precision Navigation, Mountain View, 

In a preferred embodiment, the GDM of the subject Calif.). See FIG. 4. 

invention will have a real-time operating system to coordi- Another port is used for communication using a Digital 

nate the essentially simultaneous collection of multiple types 30 Wireless spread spectrum which communicates at distances 

of data. of up to one mile line-of-sight. The wireless packet radio 

The connections and interfaces of the various components link uses the tcp/ip protocol for compatibility with sending 

of the GDM of the subject invention can be designed and and receiving data over the Internet. See FIG. 12. 

produced by one skilled in this art having the benefit of the The graphical user interface can be based on 

teachings provided herein. 35 G-WINDOWS (G -WINDOWS and G-VIEW Developers 

Following are examples which illustrate procedures for Manual, 1995, Gespac S. A., Geneva). G-WINDOWS can 
practicing the invention. These examples should not be run completely from ROM and provides an effective inter- 
construed as limiting. face. G-VIEW, a widget builder, is used for constructing 
c?v t? object-oriented widgets to control the data sensors. The 
EXAMPLE 1 40 touch-sensitive color screen is used to display color and 
Handheld Portable GDM black-and-white digital images of objects under study and is 

used in the object documentation process. Objects are digi- 

In a specific embodiment, referring to FIG. 1, the GDM tized by translating the GPS location of the data mapper to 

of the subject invention can run MICROWARE'S OS-9 me object. The distance is calculated from the user touching 

(OS-9 Version 3.0 Documentation, 1995 Microware Sys- 45 the object of interest in the digital image and the computer 

terns Corp., Des Moines, Iowa Dibble, Peter (1994) Insights, automatically finding the same pattern in the stereo pair 

An Advanced Programmers Guide to OS-9 3.0 Edition, image. Bearing and orientation are calculated from the 

Microware Systems Corp., Des Moines, Iowa; Astec DC to compass and inclinometer reading. 

DC converter, ^tec America Inc., Oceanside, Calif ; and When m voice and control mode> me tem 

Dayan, Paul S. (1992^^ 50 verba Uy prompts the operator for input and flashes the 

Industrial Limited^Durham, UK) embedded operating sys- appropriate icon when ^ voice comma nd from the user is 

LTJo^A^™ 6 h ™,oL l o e ? mpUte i 15 a SSfo Processed. This leaves the operator's hand free for other 

SB o?f i°e 5 ° ? WSSSS "i°c ^ W P0WCr CM ° S tasks. The voice command and control also aids in poor 

MC 8340 Smgle Board System, 1995, Gespac Inc., Mesa, visibility situations and for users ^th visual handicaps. 
Ariz.; and Gespac Quad Asynchronous Serial Interfaces 55 

XSBSIO-2, 1995, GESPAC Inc., Mesa, AZ) with two mega- EXAMPLE 2 
bytes of ROM and one megabyte of static RAM. It has a 

MOTOROLA 68340 CMOS low power processor 1 with six Measurement of Remote Objects 

serial ports. See FIG. 3. Objects of interest can be measured in the field by 

A small carrier board holds two industry pack modules. 60 touching two points on one picture and letting the system 

The first has a GREENSPRINGS IP-LCD VGA calculate a three dimensional coordinate for each point from 

(Greenspring Computer IP-LCD VGA Industry Pack, 1995, the direction, pitch, and roll data, lens characteristics and 

Greenspring Computing Inc., Menlo Park, Calif.) controller lens spacing. Distance between points can be calculated. All 

52 interfaced with a small SHARP 640x480 TFT active of the data is stored with the stereo pair of images for future 

matrix flat panel color screen. The color screen 54 Sharp 65 processing with similar software on a desktop computer 

LQ64D142 TFT-LCD Module, 1995, Sharp Liquid Crystal and/or if the user desires to capture more detailed informa- 

Display Group, 1995, Camas, Wash, displays digital images, tion from the photo. 
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EXAMPLE 3 

Three Dimensional Digitizing of Objects 

As the user points the data collector unit in the direction 
of an object of interest the GPS is providing location 
coordinates, the system is also monitoring the direction pitch 
and roll of the stereo pair of cameras. Once the relative three 
dimensional coordinate for the object under study is calcu- 
lated from the stereo data and the three dimensional heading 
of the camera the actual location of the object is calculated 
and recorded. 

EXAMPLE 4 

Mapping Objects in the Field 

The coordinate derived for the objects located in the 
stereo pairs of images is used to construct a geographic 
information system map layer. Each object can have 
attribute data stored with it including a photo of the object, 
its dimensions, and other attribute data the user desires to 
include. For example a verbal description can be digitized by 
the voice processor for a tag attribute or an icon driven 
system can be used to capture field observations. 

EXAMPLE 5 

Special Processing of Pictures 

Each stereo pair of pictures has location and directional 
data stored with it thus the plane of the photograph is known. 
This makes it possible to correct the picture for orientation. 
For example if the picture was taken at an angle the image 
can be rotated to remove the tilt in the picture. Also a floating 
scale is placed on the surface of an object in a picture so 
dimensions can be direcdy read off by the user. The units on 
the scale are automatically set from the information obtained 
from processing the stereo images and their associated data. 

EXAMPLE 6 

Field Correction of Existing Geographic 
Information System Data 

Existing data is corrected in the field by loading an 
existing map data set on to the system. The user takes the 
map into the field. As features on the map are photographed 
the user touches the object on the map and the same one in 
the stereo pair of pictures. The system automatically calcu- 
lates the displacement to the object and translates the GPS 
coordinate of the handheld data mapper to the object. This 
new accurate coordinate is then used to update the old 
feature coordinates including adding an altitude coordinate. 

EXAMPLE 7 

Generation of Three Dimensional Wire Frame 
Sketches 

Several algorithms are used to enhance the edges of 
objects of interest. For example the lapse transform of an 
image will highlight the edges of an object in the picture. 
These algorithms are used to assist the calculation of the 
displacement measurement of the image pairs of the same 
objects in the stereo pair of digital photographs. Once the 
images are highlighted the system can then generate a wire 
frame three dimensional model for the object under study. In 
complex situations the user could take several pairs of 
images from different sides of the object under study. A more 
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complete digital three dimensional map and model of the 
object is then constructed from the data. 

Example 8 

5 

GDM Utilizing the Motorola 821 Power PC 
Processor 

Referring to FIG. 13, in another specific embodiment of 
the subject invention, a GDM was developed around the 

10 highly integrated MOTOROLA 821 Power PC processor. 
This microprocessor has interfaces to PC-card (PCMCIA), 
RAM memory, flash memory, and LCD display, as part of 
the chip. The GDM PC-card, RAM memory, flash memory, 
and LCD display are connected directly to the 821 PPC chip 

15 via the respective built in controllers and logic. Some of the 
advantages of using this processor are low power, small size, 
and lower cost. Many types of devices can be inserted in the 
PC-card slot however the most frequent use is to use a solid 
state hard disk drive to store maps, digital images, and 

20 associated data. 

The MOTOROLA MPC821 also has a six-channel high 
speed communication processor. The first channel (SCC1) is 
connected to an Ethernet transceiver which connects directly 

25 to Ethernet local area networks for in office use. The second 
communication port (SCC2) provides a five megabit syn- 
chronous interface to the two DYCAM Gator digital cam- 
eras which have a high speed synchronous port provided by 
their integral digital signal processor (TEXAS INSTRU- 

30 MENTS TMS320cx). The third port (SMC 1) is switched 
between providing a local serial port and the 2.4 gigahertz 
spread spectrum two-way digital packet radio. The fourth 
port (SMC2) is interfaced to a TRIMBLE eight-channel 
GPS receiver. The fifth channel serial peripheral interface 

35 (SPI) is connected to the touch screen controller which 
provides for user input by touching an LCD panel with a 
stylus or a finger. The sixth communication port uses the 
12C interface (12Q. I2C provides a simple internal network 
which provides interface to the DURACELL DR35 smart 

40 battery's 12C port, and the 12C serial interface connected to 
the PRECISION NAVIGATION digital compass and incli- 
nometer. When a second GPS receiver is used, the I2C also 
provides a convenient interface. 

The core software for the GDM computer is provided by 

45 the MICROWARE OS9000 real time operating system for 
the Power PC. Device drivers support the interface to all the 
components attached to the communication channels as well 
as the LCD graphic display and the PC-card slot. TCP/IP 
network software provides wired and wireless network 

50 interface to the Internet. Sun Microsystems JAVA virtual 
machine is used to enhance the network features of the 
GDM. Java classes are provided for each of the major 
devices which make up the GDM. For example there is a 
camera class, a compass and inclinometer class, a battery 

55 class, and a GPS class. Each of these classes have JAVA 
native methods written in C or C++ to provide low level 
access to the data sensors and to do some processing. FIG. 
14 is a flow diagram of the Camera Class. All of the graphics 
interface the user sees is written in JAVA, and can be 

60 download over the network connection. For a GDM appli- 
cation to take a picture, the application first uses the Camera 
class to create a camera object which then can be passed 
messages. For example to capture a stereo pair of digital 
images the "takePicture" message is sent to the camera 

65 object. 

FIG. 15 provides a representation of a hand-held version 
of the GDM which is implemented with the system 
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described above. In FIG. 15, stereo cameras and stereo GPS 
receivers are shown. The GPS antennas are mounted at right 
angles to the lenses on a platform which is attached to the 
core hand-held computer module at angle a, preferably 
approximately 150 degrees. This design makes it more 
convenient for the user to view images on the screen (live 
view finder) while taking a picture. FIG. 16 illustrates an 
embodiment of the subject GDM where the stereo GPS and 
stereo cameras are mounted on a structure which is separate 
from the user interface. This embodiment is useful in 
situations where the GDM is used with a vehicle. The 
structure housing the GPS antennas and cameras can be 
mounted to the exterior of the vehicle and t he comput er 
jispJajUP Qdylc can be mountedloThe' lfltedbr ofthe vehicle 
or designed for r iand-neld use. ' "~ 

It shoulcTbe understood triat The examples and embodi- 
ments described herein are for illustrative purposes only and 
that various modifications or changes in light thereof will be 
suggested to persons skilled in the art and are to be included 
within the spirit and purview of this application and the 
scope of the appended claims. 

What is claimed is: 

1. A device for gathering position data, comprising: 

a position means for providing the position of said device, 
an imaging means for imaging an object, wherein said 
imaging means comprises two digital cameras, wherein 
processing data from both cameras provides the dis- 
tance from said device to an imaged object, and 
a processing means, wherein said processing means can 
receive and process data from the position means and 
data from the imaging means in order to provide 
position data for an imaged object. 

2. The device, according to claim 1, wherein said position 
means comprises a receiver based on a satellite positioning 
system. 

3. The device, according to claim 2, wherein said position 
means comprises at least one GPS receiver. 

4. The device, according to claim 3, wherein said position 
means comprises at least two GPS receivers. 

5. The device, according to claim 1, wherein said position 
means is a receiver based on a ground based positioning 
system. 

6. The device, according to claim 1, further comprising a 
heading means for providing a heading of said device. 

7. The device, according to claim 6, wherein said heading 
means is a digital compass. 

8. The device, according to claim 6, wherein said heading 
means is at least two GPS receivers. 

9. The device, according to claim 1, further comprising a 
user interface means for user interaction with the device. 

10. The device, according to claim 1, further comprising 
a pitch means for providing a pitch of said device. 

11 . The device, according to claim 10, wherein said pitch 
means is an inclinometer. 

12. The device, according to claim 10, wherein said pitch 
means of said device is a gyroscope. 

13. The device, according to claim 10, wherein said pitch 
means is at least two GPS receivers. 

14. The device, according to claim 10, further comprising 
a roll means for providing a roll of said device. 

15. The device, according to claim 14 wherein said roll 
means for providing a roll of said device is an inclinometer. 

16. The device, according to claim 14, wherein said roll 
means is a gyroscope. 

17. The device, according to claim 14, where said roll 
means is at least two GPS receivers. 

18. Trie device, according to claim 9, wherein said user 
interface means is selected from the group consisting of a 
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flat panel display with touch screen, voice interaction, pen 
commands, and a keyboard. 

19. The device, according to claim 1, comprising at least 
three GPS receivers, wherein data from said three GPS 

5 receivers provides a position, a heading, a pitch, and a roll 
of said device. 

20. The device, according to claim 19, wherein data from 
said two cameras combined with data from the three GPS 
receivers provides three dimensional position data of an 

Q imaged object. 

21. The device, according to claim 1, further comprising 
a means for wireless communication with a remote host 
computer. 

22. The device, according to claim 1, wherein said pro- 
cessing means processes position data of imaged objects 

15 such that the position relation of two imaged objects is 
provided. 

23. The device, according to claim 1, further comprising 
at least one gyroscope, wherein said gyroscope provides 
data related to information, selected from the group consist- 

20 ing of heading of the device, pitch of the device, and roll of 
the device, and wherein said gyroscope can be calibrated 
from data received from a GPS satellite while said device is 
moving. 

24. The device, according to claim 1, wherein said device 
25 is incorporated with a vehicle. 

25. The device, according to claim 24, wherein imaging 
means can be removed from the vehicle while said position 
means remains affixed to the vehicle, and wherein said 
device further comprises a means for providing data with 
respect to the imaging means' position relative to the 
position means' position, such that the position of the 
imaging means can be determined from the position of the 
position means and the data with respect to the imaging 
means' position relative to the position means* position. 

26. The device, according to claim 25, wherein said 
35 means for providing data with respect to the imaging means' 

position relative to the position means' position comprises at 
least one gyroscope. 

27. The device, according to claim 24, wherein said 
device comprises at least two GPS antennas, at least two 

40 cameras, and at least one gyroscope mounted to the vehicle. 

28. The device, according to claim 26, wherein once the 
vehicle is position in a region where the device's position 
means functions correctly such that the device's position is 
determined, the gyroscope can be calibrated and the imaging 

45 means can then be used in regions where the position means 
does not correctly function. 

29. The device, according to claim 25, wherein said 
means for providing data with respect to the imaging means' 
position utilizes dead reckoning position calculation. 

50 30. The device, according to claim 1, wherein said imag- 
ing means is based on a non-visible portion of the light 
spectrum. 

31. The device, according to claim 1, further comprising 
at least one sensor selected from the group consisting of: a 

55 chemical sensor, a heat sensor and an environmental sensor. 

32. The device, according to claim 1, wherein said imag- 
ing means comprises at least three cameras. 

33. The device, according to claim 1, wherein at least one 
of said two cameras is movable such that the distance 

60 between the cameras can be adjusted. 

34. The device, according to claim 1, wherein images 
produced by the cameras are compared to determine a 
distance apart the imaged object is in the images produced 
by the cameras, and wherein the distance apart the imaged 

65 object is in the images produced by the cameras is used to 
determine the distance from said device to the imaged 
object. 
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35. The device, according to claim 34, wherein the 
distance from said device to the imaged object is determined 
from the distance apart the imaged object is in the images 
produced by the cameras according to the following equa- 
tion: 



4> = 



10 



where d 0 is the distance to the imaged object, d c is a 
spacing between cameras, d, is the distance apart the 
imaged object is in the images produced by the 
cameras, and f is the focal length of the cameras lenses. 



18 



36. The device, according to claim 34, wherein enhance- 
ment of the imaged object's edges is utilized in the deter- 
mination the distance apart the imaged object is in the 
imaged produced by the cameras. 

37. The device, according to claim 1, wherein said posi- 
tion means comprises a map display, wherein a user desig- 
nates the location of said device with respect to the map 
display. 

38. The device according to claim 1, wherein said cameras 
are rotated such that each points toward the object. 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 

PATENT NO. : 6,083,353 

DATED : My 4, 2000 

INVENTOR(S) : 

John F. Alexander, Jr. 

It is certified that error appears in the above-identified patent and that said Letters Patent is hereby 
corrected as shown below: 

Column 7, line 30: "GPS" should read -GPS 32-. 
Column 9, line 18: "10 optics" should read -optics 10-. 
Column 11, line 52: "Gespac" should read -GESPAC-. 
Column 14, line 30: "gigahertz" should read -gigaHertz--. 

Column 14, line 39: "12C port, and the 12C serial" should read -I2C port, and the I2C 
serial-. 

Column 1 5, line 40 (claim 5): "means is a receiver based" should read 
-means is based-. 



Signed and Sealed this 
Twenty-ninth Day of May, 2001 

NICHOLAS P.GOD1CI 

Attesting Officer Acting Director of the United States Patent and Trademark Office 
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